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— *  Performance  appraisal  systems  play  an  important  role  in  the  effective 
functioning  of  any  large  organisation.  However,  in  spite  of  their 
relevance  and  the  increasing  demand  for  effective  performance  appraisal 
systems,  our  ability  to  improve  these  systems  seems  to  have  reached  a 
plateau.  In  order  to  advance  beyond  our  present  state  with  regard  to 
performance  appraisal,  it  is  necessary  to  have  a  better  understanding  of 
the  psychological  processes  involved  as  one  person  makes  judgments 
about  the  performance  of  another. 


The  theoretical  positions  which  have  been  developed  in  the  field  of 
social  cognition  provide  a  useful  framework  for  examining  these 
processes,  since  performance  appraisal  is  essentially  a  cognitive  task 
performed  within  the  social  and  motivational  constraints  of  a  particular 
type  of  social  situation.  Therefore,  the  purpose  of  this  one-year 
research  contract  was  to  adopt  a  process  view  to  performance  appraisal 
and  then  explore  and  test  assumptions  from  this  framework  with  empirical 
research.  Variables  that  have  been  identified  as  important  to  appraisal 
were  manipulated  and/or  measured,  in  order  to  explore  those  factors  that 
influence  the  accuracy  of  performance  appraisals.  Specifically,  the 
purpose  of  this  contract  was  to  oontinue  and  complete  research  already 


begun  under  contracts  to  Barnes-Farrell  and  Ilgen  (#N00014-82-K-0449) 
and  to  Barnes-Farrell  (IH00014-83-K-0757) .  \ 


The  bulk  of  the  work  carried  out  under  this  contract  focused  on  four 
issues:  (1)  the  development  of  procedures  and  materials  for  indexing 
the  accuracy  of  performance  assessments *  (2)  an  examination  of  the 
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processes  underlying  occupational  age-typing;  (3)  the  impact  of  gender- 
related  variables  in  the  evaluation  context  on  the  process  and  accuracy 
of  performance  appraisal;  and  (4)  the  integration  and  evaluation  of 
recent  empirical  research  which  has  examined  performance  appraisal  from 
a  process  perspective.  The  work  performed  relevant  to  each  of  these 
issues  is  briefly  summarised  below. 

Indexing  Accuracy 

A  time-consuming,  but  extremely  important,  project  for  this  contract 
was  the  completion  of  the  development  and  testing  of  laboratory 
materials  that  would  allow  us  to  index  the  accuracy  with  which 
performance  measurements  are  made.  Three  types  of  materials  were 
developed:  videotaped  samples  of  waiter /waitress  performance,  with 
accompanying  measurement  instruments;  stimulus  materials  for  the 
controlled  presentation  of  distributions  of  performance  for  the  job 
"field  interviewer”;  and  paper-and-pencil  measures  for  the  evaluation  of 
performance  in  a  variety  of  gender-typed  occupations. 

Videotapes.  The  most  extensive  and  ambitious  effort  undertaken  for 
this  project  was  the  development  of  a  set  of  videotaped  vignettes  of 
performance  for  the  occupation  "waiter/waitress".  Production  of  the 
videotapes  and  the  accompanying  BARS  scales  used  to  index  the  accuracy 
of  performance  assessments  was  oompleted  under  previous  contracts,  and 
is  described  in  detail  in  Barnes-Far r ell,  L'Heureux-Barrett  ft  Kealy 
(1986).  During  the  current  contract,  these  materials  were  used  to 
produce  interactive-video  programs  intended  to  fulfill  two  purposes. 
First,  inter active- video  technology  provides  an  efficient  means  of 
controlling  and  presenting  known  stimulus  materials  to  subjects,  and  for 
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|  recording  their  assessments  of  performance.  Programs  allowing  us  to 

|  interface  our  videotaped  stimulus  materials  with  the  computer  allow  us 

to  select  and  control  the  order  and  amount  of  performance  information 
that  subjects  encounter  when  they  are  given  the  task  of  appraising 
worker  performance.  They  will  be  of  continuing  value  to  us  as  we 
continue  to  explore  the  impact  of  various  aspects  of  the  appraisal 
[  context  and  the  appraisal  task  on  the  accuracy  of  performance 

measurement.  Second,  this  approach  represents  a  very  practical 
application  of  our  research  materials  for  the  purpose  of  training 

i 

|  appraisers  to  improve  the  quality  of  their  performance  assessments.  The 

rationale  and  procedures  which  were  used  to  develop  our  prototype 

i 

interactive  training  program  are  described  in  Kealy,  Barnes-Far r ell  & 
L'Heureux-Barrett  (1987) . 

Distributional  Measurement.  A  second  approach  to  presenting 
appraisers  with  realistic,  but  known,  samples  of  worker  performance  is 
the  distributional  measurement  approach  developed  and  endorsed  by  Kane 
(cf.  Kane,  1980).  Materials  originally  developed  by  Dr.  Kane  for  the 
evaluation  of  "field  interviewer”  performance  were  revised  and  pretested 
under  this  contract.  Although  there  appeared  to  be  some  problems 
initially  with  one  of  the  performance  dimensions  being  manipulated  with 
these  materials,  these  problems  have  been  resolved,  and  we  have  now 
produced  stimulus  materials  which  will  allow  us  to  simulate  actual 
distributions  of  worker  performance  over  extended  periods  of  time, 
rather  than  tapping  into  performance  at  a  single  point  in  time,  as  is 
typically  done  in  appraisal  research. 
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Gender-typed  Occupations.  The  final  group  of  stimulus  materials 
produced  for  this  contract  is  a  collection  of  paper-and-pencil  measures 
which  describe  behaviors  associated  with  known  levels  of  performance  for 
a  variety  of  male-dominated  and  female-dominated  occupations.  These 
materials  are  accompanied  by  behaviorally  anchored  rating  scales 
designed  to  measure  work  performance  on  the  underlying  performance 
dimensions  represented  by  these  behavioral  descriptions.  Portions  of 
the  development  work  on  these  materials  were  carried  out  in 
collaboration  with  Drs.  Jeanette  Cleveland  and  Kevin  Murphy  of  Colorado 
State  University.  The  stimulus  materials  were  subsequently  used  to 
conduct  two  studies  which  are  described  later  in  this  report,  and  are 
currently  being  utilized  in  additional  studies  of  the  appraisal  process. 

Occupational  Age-typing. 

Two  studies  were  conducted  which  addressed  the  processes  underlying 
age-typing  in  work  organizations.  Both  studies  have  implications  for 
understanding  the  conditions  under  which  age  discrimination  is  likely  to 
occur.  As  such,  they  address  one  possible  source  of  contamination  in 
the  assessment  of  worker  performance.  First,  a  pair  of  laboratory 
experiments  by  Barnes-Farrell  (1985)  investigated  the  evidence  for  the 
existence  of  occupational  age-typing.  Barnes-Farrell  argued  that  age¬ 
typing  may  be  the  logical  outcome  of  the  process  of  matching  age-related 
social  stereotypes  of  workers  with  our  expectations  about  the  skills 
required  by  different  occupations.  The  results  of  these  experiments 
support  her  contention  that  some  occupations  are  age-typed,  and  that  the 
"age-type"  of  an  occupation  is  closely  associated  with  the  types  of 
skills  thought  to  be  required  by  that  occupation.  A  second  study,  by 
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Barnes-Farrell  and  L'Heureux-Barrett  (1986) ,  was  designed  to  investigate 
the  implicit  personality  theories  that  adults  hold  about  the  work- 
related  characteristics  associated  with  workers  of  different  ages. 
Although  a  number  of  studies  have  investigated  social  stereotypes  that 
our  society  holds  about  the  elderly,  we  have  very  little  empirical  data 
about  our  perceptions  of  older  workers.  In  particular,  it  is  important 
to  identify  our  expectations  about  the  relationship  between  worker  age 
and  work-related  skills.  In  this  field  survey,  we  were  able  to 
operationalize  age  stereotypes  in  terms  of  profiles  of  the  work-related 
skills  believed  to  be  held  by  workers  in  different  age  groups.  The 
profiles  provide  a  means  of  identifying  the  kinds  of  occupations  that 
are  likely  to  be  of  particular  concern  to  employers  because  of  the 
increased  potential  for  age  discrimination  in  personnel  decisions 
concerning  workers  in  these  occupations. 

Gender-related  Variables  and  the  Appraisal  Process. 

Barnes-Farrell  and  L'Heureux-Barrett  (1987)  conducted  a  laboratory 
investigation  which  focused  on  the  effects  of  job  and  task  gender-typing 
on  rating  accuracy.  It  was  found  that  the  accuracy  of  ratings  of  task 
performance  was  significantly  affected  by  the  interaction  between  the 
gender-type  of  the  task  and  the  gender- type  of  the  job  in  which  the 
worker  was  employed,  although  worker  gender  did  not  affect  either  the 
level  or  the  accuracy  of  performance  evaluations.  The  reasons  that 
performance  information  in  task  areas  which  are  congruent  with  the 
gender-type  of  the  job  is  evaluated  more  accurately  are  unclear.  One 
plausible  explanation  is  that  the  most  elaborate  category  structures 
accessible  when  evaluating  workers  in  female-typed  occupations  will  be 
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those  that  are  closely  associated  with  the  female  sex  role  stereotype. 
Similarly,  the  accurate  evaluation  of  performance  in  male-typed  task 
areas  may  be  be  easier  to  accomplish  in  the  context  of  male-typed 
occupations  because  the  job  context  increases  the  accessibility  of  male- 
typed  categories.  This  interpretation  is  consistent  with  suggestions 
made  by  a  number  of  researchers  regarding  the  cognitive  processes  which 
lead  to  accurate  appraisal  of  social  information.  A  second  study  by 
Barnes-Farrell  and  L'Heureux-Barrett  (1987)  examined  the  manner  in  which 
task  evaluations,  once  made,  are  integrated  to  form  overall  evaluations 
of  employee  work  performance.  It  was  expected  that  the  manner  in  which 
performance  information  is  weighted  and  combined  may  also  be  affected  by 
gender-related  variables.  Of  particular  interest  in  the  study  was  the 
way  in  which  worker  gender  might  affect  the  relative  weighting  of 
positive  and  negative  performance  information.  A  significant 
interaction  of  performance  level  and  target  sex  was  observed,  suggesting 
that  performance  information  is  integrated  differently  for  male  workers 
and  female  workers.  The  results  of  these  two  experiments  demonstrate 
that  there  are  a  number  of  ways  that  gender  may  play  a  role  in  the  way 
we  carry  out  the  cognitive  tasks  which  underly  the  accurate  appraisal  of 
employee  work  performance.  These  go  beyond  the  direct  effects  of  worker 
gender  and  the  interactive  effects  of  worker  gender  and  occupation  on 
the  level  of  performance  ratings.  The  gender-related  effects  of  the 
context  in  which  evaluations  of  behavioral  information  are  made,  and  on 
the  manner  in  which  such  evaluations  are  combined  suggest  that  sex 
discrimination  in  the  level  of  performance  ratings  and  the  allocation  of 
rewards  may  be  only  the  most  visible  outcomes  of  the  way  we  view  men  and 


women  at  work 
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Evaluation  of  Current  Research  on  Performance  Rating  Accuracy. 

A  final  product  supported  by  this  research  contract  is  an  integrative 
literature  review  which  represents  a  collaboration  between  this  contract 
and  a  similar  program  of  research  conducted  by  Dr.  Daniel  Ilgen  of 
Michigan  State  University  (Ilgen,  Barnes-Farrell  &  McKellin,  1986) .  Our 
purpose  was  to  examine  the  research  which  has  been  generated  by  the 
recent  refocusing  of  the  performance  appraisal  field  on  cognitive 
processes.  The  review  focuses  on  the  relative  contributions  of  this 
"change  of  course”  to  our  understanding  of  accurate  performance 
assessment,  and  to  the  identification  of  interventions  and  systems 
designs  which  may  facilitate  accurate  appraisal.  Specifically,  a 
conceptual  framework  for  performance  appraisal  front  the  appraiser's 
perspective  is  presented,  and  the  empirical  research  between  1980  and 
1986  is  evaluated  as  it  relates  to  that  framework. 
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